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A  series  of  experiments  addressed  the  adequacy  of  act  generation 
performance^  an  important  precursor  to  problem  structuring.  Each  of  two 
decision  problems  was  studied  by  a  series  of  three  experiments.  In  the 
first  es^riment,  subjects  were  givei  a  realistic  decision  problem  and 
asKed  to  respond  with  ar^  act  occurring  to  them.  In  the  second  experiment, 
the  acts  suggested  were  evaluated  1:^  different  subjects  for  feasibility. 
In  a  third  experiment,  additional  subjects  estimated  the  utility  of  the 
acts  judged  feasible.  Die  act  generaticxi  performance  of  subjects  was 
evaluated  using  two  techniques.  First,  a  decision  tree  was  generated  by 
the  experimenters  by  combining  the  acts  suggested  by  edl  subjects.  The 
decision  tree  generated  ty  each  subject  was  compared  with  the 
experimenter-generated  tree.  It  was  found  that  subjects  fedled  to  generate 
important  limbs  and  branches  of  the  group  decision  tree.  Second,  the 
quality  of  the  trees  generated  ty  individued  subjects  was  evcQuated  by  em 
opportunity  loss  calculation.  This  calculation  provided  an  estimate  of  the 
potential  cost  of  failing  to  generate  limbs  and  branches  of  the  decision 
tree.  The  c^iportunity  loss  analysis  suggested  that  the  failure  to  generate 
a  complete  tree  could  be  costly. 


An  Evaluation  of  Human  Act  Generation  Performance 

Several  specific  processes  occur  prior  to  making  a  carefully^-oonsidered 
decision.  First,  a  decision  maker  must  recognize  that  a  problem  exists  for 
whi<±  a  decision  is  required.  Next,  relevant  components  or  dimensions  of 
the  problem  are  identified.  These  components  include  1)  acts  which  may 
potentially  lead  to  the  goal,  2)  states  of  the  world  (hypotheses)  vdiich 
may  influence  the  outcomes  of  these  acts,  and  3)  possible  consequences  or 
outcomes  of  acts.  These  processes,  and  others,  are  included  in  the  general 
term  "predecision  processes."  Predecision  processes  are  important 
precursors  to  the  actual  decision,  and  can  have  a  major  inqpact  on  the 
quality  of  the  decision  (Hunphreys,  1979) . 

In  this  paper  we  concentrate  on  cxie  of  the  predecison  processes,  act 
generation.  Act  generation  is  the  process  of  generating  options  for  action 
available  to  the  decision  maker.  Oliese  acts,  designed  to  achieve  the 
desired  goal,  are  the  "menu"  of  options  from  which  the  decision  maker  must 
eventually  choose.  It  is  quite  iiiportant  that  this  menu  be  as  complete  as 
possible,  to  offer  the  decision  maker  an  effective  course  of  action.  If 
the  list  of  acts  generated  1;^  t)»  decision  maker  is  inoompQ.ete,  the 
subsequent  decision  may  not  be  the  best  which  could  be  made  ixider  the 
circumstances,  leading  to  possible  "opportunity  loss." 

The  investigation  of  human  act  generation  capablilites  has  received  scant 
attention  to  date.  The  lack  of  concern  with  act  generaticxi,  and 
predecision  processes  in  general,  is  probably  due  to  several  historical 
factors.  These  topics  initially  appeared  to  be  intractable  for  several 
reasons.  First,  early  research  on  hunan  decision  making  compared  hunan 
performance  with  optimal  models.  As  there  were  no  optimal  models  for 
predecision  processes,  the  tcpic  was  largely  ignored.  Raiffa  (1968) ,  for 
example,  stated  that  elicitation  of  structure  was  an  important  problem  he 
"wanted  to  duck."  The  recent  emergence  of  a  cognitive  approach  to  decision 
making  (Wallsten,  1980)  has  broadened  the  areas  of  inquiry  to  include  All 
aspects  of  decision  making,  not  just  tractable  candidates  for  normative 
modeling.  Second,  even  though  predecision  processes  are  legitimate 
candidates  for  study  from  a  cognitive  perspective,  the  processes 
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themselves  afpeared  to  be  intractable  topics  of  stud^  whidi  would  not 
reveal  their  secrets  easily. 

There  has  been  some  research  on  hypothesis  generation,  which  is  a  process 
relevant  to  act  generation  (Gettys  and  Fisher,  1979;  Manning,  Gettys, 
Nicewander,  Fisher,  &  N^e,  1960;  M^e,  Gettys,  Manning,  Baca  &  Fisher, 
in  press) .  The  major  conclusion  of  these  and  other  studies  was  that 
hypothesis  generation  perfommnce  is  largely  dependent  oi  the  success  of 
accessing  information  available  in  memory.  Subjects  failed  to  retrieve 
rnarv  important  hypotheses  from  memory,  yet  when  asked  to  assess  the 
oonpleteness  of  their  retrieval,  th^  grossly  over-estimated  it  (Hehle, 
et.  al. ,  in  press) .  Hus  series  of  studies  suggests  similar  deficiencies 
in  accessing  acts  stored  in  memory  will  be  found,  due  to  the  similarity  of 
act  and  hypothesis  generaticxi. 

Pits,  Sadis  and  Heerboth  (1980)  investigated  methods  of  eliciting  acts 
from  subjects  in  the  first  study  cn  the  topic  of  act  generation  in  a 
decision  theory  ccxitext.  Th^  found  serial  presentation  of  problem 
objectives  or  goals  enhanced  act  generation  performance  as  compared  to 
conditions  where  subjects  were  given  either  exen^ar  acts  or  all  problem 
objectives  simultaneously.  They  also  observed  that  subjects  did  not  seem 
to  generate  a  very  complete  set  of  acts,  averaging  less  than  a  third  of 
the  acts  the  experimenters  thought  were  "worth  considering.”  However, 
subjects  had  limited  time  to  respond,  and  their  performance  might  have 
been  better  had  they  been  allowed  unlimited  time. 

Decision  problem  structuring  is  the  process  of  explicitly  modeling  a 
decision  problem  for  purposes  of  Decision  Analysis.  It  is  curious  that 
ccxisiderable  effort  has  been  devoted  to  enhancing  act  generation 
performance  (and  other  predecisicxi  processes)  given  the  paucity  of 
information  on  the  quality  of  unaided  hunan  problem  structuring. 
Nevertheless,  such  efforts  exist  (e.g.  Leal  and  Pearl,  1977;  Pearl,  Leal  & 
Saleh,  note  1;  Weiss  &  Kelly,  note  2;  Merkhofer,  Miller,  Robinson  & 
Korean,  note  3) .  The  work  of  Pearl  and  his  colleagues  is  of  particular 
interest  because  they  speculated  (xi  the  cognitive  mechanisms  involved  in 
act  generation. 


Given  the  near  absence  of  research  on  hunan  act  generation  and  the 
considerable  interest  in  inproving  act  generation,  it  would  seem  desirable 
to  investigate  himan  act  generation  capabilities.  The  present  stwfy 
attempts  provide  a  preliminary  examinaticxi  of  act  generation,  one  that 
will  establish  the  basis  for  a  study  of  the  cognitive  processes  involved. 
TWO  major  questions  will  be  addressed: 

1.  what  extent  subjects  generate  ^  *>«•*>  set  of  acts? 

2.  Shat  Is  j^be  potential  ato.  failures  generate  important  acta? 

The  first  of  these  questions  is  important  because  the  Pitz,  et.  al.  stu^ 
addressed  the  question  of  capability  only  incidentally  to  its  main 
purpose — comparing  various  act  elicitation  techniques.  They  used  no 
criterion  of  performance,  and  allowed  only  a  limited  ennunt  of  time  for 
each  problem.  Thus,  their  data  are  less  than  definitive.  More  basic 
information  on  act  generation  performance  is  needed.  Can  subjects  generate 
most  of  the  important  acts  to  solve  a  problem,  or  is  their  unaided 
performance  so  deficient  that  even  more  attention  should  be  given  to 
aiding  act  generation  for  important  decisions? 

The  importance  of  the  second  question  becomes  clear  when  it  is  realized 
that  it  is  unnecessary  (except  perhaps  from  a  Utopian,  normative  view)  for 
the  decision  maker  to  generate  aU  important  acts.  It  is  necessary  for  at 
least  one  satisfactory  act  to  be  generated,  and  it  me^  be  sufficient  if 
only  a  few  very  attractive  acts  are  on  the  "maiu''.  After  fulfilling  the 
above  requirements,  the  practical  importance  of  failing  to  generate  all 
acts  roey  be  small.  Practical  decision  makers  may  "satisfice",  choosing  the 
first  act  meeting  some  minimun  criterion  of  acceptability  (Simon,  1956). 
Even  if  they  "optimize"  by  generating  a  number  of  acts  then  picking  the 
best,  the  difference  between  their  best  act  and  the  theoretically  optimum 
act  may  be  small.  Therefore,  a  second  interesting  question  regarding  act 
generation  performance  is  the  cost  of  failing  to  generate  the  optimum  act, 
or  the  "opportinity  loss"  (Raiffa,  1968)  due  to  deficiencies  in  act 
generation.  Even  thou^  subjects  fail  to  generate  the  "optimal"  act,  they 
may  generate  another  act  having  a  similar  utility.  In  this  case,  failure 
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to  generate  a  oorplete  menu  of  ac±s  would  be  of  little  Inqportance.  If, 
however,  the  difference  In  utility  betweoi  the  best  act  generated  and  the 
optimal  act  Is  large,  then  there  would  be  cause  for  concern. 

Question  1  Is  addressed  by  an  experiment  In  which  subjects  were  given  a 
realistic  decision  problem  and  were  asked  to  generate  any  act  that  ml^t 
conceivably  solve  the  problem.  The  second  question  Is  addressed  In  a 
second  section  of  the  paper  by  a  series  of  two  experiments.  In  the  first 
e]q)eriinent  of  the  series,  acts  generated  by  previous  subjects  were 
screened  for  feasibility.  The  final  experiment  obtained  utility  estimates 
for  the  feasible  acts.  These  estimates  permitted  calculation  of 
"opportunity  loss". 

h  critetion  ior  act  aencraticm  performance 

The  seemingly  intractable  problem  in  a  study  of  act  generation  capability 
is  the  difficulty  in  creating  a  criterion  of  performance.  TVo  major 
2^roadies  can  be  used  to  create  such  a  criterion.  First,  one  can  refer  to 
authoritative  sources  (Fischhoff,  Slovic,  and  Lichtofistein,  1978)  such  as 
an  "e]q)ert's"  structure  of  the  act  generation  problem.  While  this  may  be 
an  excellent  approach  when  it  can  be  utilized,  no  such  authoritative 
sources  exist  for  most  interesting  decision  problems. 

A  second  approach  does  not  rely  on  expert  judgment,  but  instead  uses 
ordinary  subjects'  responses.  In  this  procedure,  described  in  more  detail 
in  Gettys,  Mehle,  Baca,  Fisher,  and  Manning  (note  4) ,  the  reeponses  of  a 
groip  of  subjects  are  pooled  to  create  a  lower-bound  estimate  of  the 
number  of  possible  acts.  A  group  menu  of  acts  which  mi^t  successfully 
solve  the  problem  is  created  by  taking  the  union  of  the  acts  suggested  by 
each  individual.  The  result  is  a  lowerHoound  estimate  in  that  a  few  more 
potential  acts  would  be  discovered  if  more  subjects  were  added  to  the 
group.  This  approach  has  the  advantages  that  it  can  be  applied  to  a  wide 
varied  of  situations,  and  it  is  a  conservative  criterion  of  performance 
because  it  is  a  lower-bound  estimate.  Its  major  disadvantage  is  that  it  is 
necessary  to  add  maty  subjects  before  the  pool  of  acts  ceases  to  grow. 
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Experimait  1 


Method 

Criteria  £qc  selecting  problems.  In  cdioosing  act  generation  prohlens  we 
were  concerned  with  the  issue  of  the  possible  interaction  between 
e3g«rtise  and  the  performance  of  the  subjects.  Ideally,  expert  populations 
might  be  preferred  ever  college  student  populations,  but  the  pooling  and 
other  prooecUres  employed  required  large  numbers  of  subjects.  We  used  over 
500  subjects  in  these  experiments  and  related  projects,  and  would  have 
found  it  difficult  to  find  even  50  expert  subjects.  Our  strategy  was  to 
attempt  to  find  meaningful  decision  problems  for  our  subjects —  problems 
for  Which  they  were  suitably  "expert”  so  our  results  could  be  generalized 
to  other  populations.  Related  work  on  l^pothesis  generation  has  suggested 
expert  subjects  and  college  stud^ts  have  similar  deficiences  in  accessing 
information  in  memory  (Gettys,  et.  al. ,  note  4;  M^le,  note  5). 

The  problems  were  chosen  with  the  following  criteria  in  mindt 

1.  The  problems  should  be  (xies  with  which  the  subjects  are  kncwledgable. 

2.  The  problems  should  be  "open-^ded"  in  that  possible  acts  should  not  be 

easily  enumerable,  or  generated  by  ai^  mechanical  rule. 

3.  The  problems  should  be  tractable  in  the  sense  that  the  number  of  acts 
generated  by  the  subjects  should  not  be  so  large  as  to  be  unroanagable. 

4.  The  problems  should  resemble  problems  encountered  in  everyday  life, 
rather  than  being  highly  artificial  problems  such  as  those  used  by  some 
researchers  in  problem  solving,  (e.g.  "Missionaries  and  Cannibals"  or 
"Towers  of  Hanoi") . 

Prnhifima.  TWO  act  generation  problems  were  used  in  this  investigation. 
After  completing  data  collection  for  the  first  problem,  we  decided  the 
results  were  of  sufficient  interest  to  warrant  including  a  second  problem 
as  a  check  on  the  generally  of  the  results  of  the  first  problem.  The  two 
problems  used  were  well-suited  to  our  subject  population.  The  first 
problem,  called  "Parking",  asked  the  subjects  to  assune  they  were  members 
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of  a  hypothetical  studoit  comnittee  appointed  by  the  officials  of  the 
Universi^  to  provide  student  ir^t  on  the  University's  parking  problem. 
Like  most  major  universities,  the  University  of  Oklahoma  has  a  perennial 
parking  problem.  IMs  problem  is  discussed  frequently  in  the  student 
newspaper.  Most  students  engage  in  frequent,  if  not  daily,  hints  for  a 
parking  space.  The  subjects'  task  was  to  list  all  acts  which  could  be 
taken  by  the  University  to  reduce  the  severity  of  the  parking  problem,  nie 
text  of  the  parking  problem  is  giv^  below. 


PARKING 


It  is  difficult  to  find  a  place  to  park  at  OU.  Althoucb 
acproximately  21,000  students  are  enrolled  and  about  3500 
faculty  and  staff  are  employed,  about  half  that  many  parking 
spaced  ate  available. 


There  are  6850  parking  places  available  on  canpis  for 
faculty/staff,  connuters,  housing.  Law,  and  OOCE.  me  Lloyd 
Noble  Center  provides  an  adoLtional  3500  spaces,  yielding  a 
total  of  10,350.  One  space  for  eveiy  typ  people  dpesn't  sound 
too  oad,  but  amone  wrK>  nas  tried  to  find'  a  parking  space  at 
9:00  a.m.  knows  this  is  a  real  problem. 


e  a  member,  of  a  .  student  organization  which,  is 


trgani _ 

researching  this  problem  for  officials  of  the  University.  Your 
task  is  to  suggest  as  many  possible  solutions  to  tbe  cdranittee 
as  you  can.  These  solutions  need  not  be  ^rfect";  often  good 
solutions  are  derived  from  ideas  which  at  first  seem  silly.  The 


,  .  _  _  __  simply 

possible  solutions  which  might  be  effective. 

K,is  inportant  for  you  to  enter  all  options  which  occur  to  you. 
This  is  similar  to  "thinking  out  loud."  We  do  not  want  you  to 
or  your  options  and  only  alter  the  ones  which  you  think  are 
Lcularl^  good.  Put  down  all  ^tions  whi,ch  ^ccur  to  ^pu,  gopd 


5r  bad.  YdUr'  score  in  this  exj&rimait  is  determined  by  the 
number  of  iptions  you  generate,  not  how  good  they  are. 


The  second  problem  involved  finding  a  place  to  live  for  an  impecunious 
Canadian  friend  who  arrived  on  campus  in  such  a  state  of  financial 
disarray  that  he  lacked  sufficient  funds  to  pay  for  a  dormitory  room  or  an 
apartment.  The  text  of  the  problem  was  carefully  worded  so  conventional 
solutions,  such  as  getting  a  job  or  asking  his  parents  for  more  money, 
were  not  viable  options.  This  problem  is  called  the  "Living"  problem,  and 
its  oonplete  text  is  found  in  J^pendlx  A. 

AlbjfiCtfi*.  Subjects  were  60  introductory  psychology  studoits  from  the 
University  of  Oklahoma  tdio  participated  in  the  experiment  for  course 
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credit,  flirty  subjects  generated  acts  for  the  Parking  problem  and  30 
subjects  generated  acts  for  the  Living  problem.  Subjects  were  recruited 
who  knew  how  to  type^  and  had  to  demonstrate  they  could  type  at  least  20 
words  per  minute  to  participate  in  the  experiment. 

Instructions.  The  instructions  were  presented  by  a  computer.  Severed  major 
topics  were  covered  in  the  instructions.  After  the  subject  was  told  the 
purpose  of  the  experiment  in  general  terms,  the  importance  of  reefponding 
with  all  possible  acts  was  en^tosized  in  several  different  peuragraphs. 
Subjects  were  instructed:  "...  We  are  interested  in  how  many  alternate 
solutions  you  can  dream  up.  It  is  not  innportant  that  these  solutions  be 
the  best  solutions  to  the  problem;  we  cu:e  interested  in  even  remotely 
possible  solutions  that  occur  to  you. "  In  another  section  of  the 
instructions  a  paragraph  was  devoted  to  exhaustively  searching  memory: 
"...Don't  give  up  on  a  problem  until  you  are  sure  you  have  run  out  of 
possible  solutions.  One  way  to  avoid  giving  up  too  soon  is  to  continue  to 
search  your  memory  for  a  while  even  though  you  believe  you  have  thou^t  of 
everything.  Usually,  a  number  of  other  possibilities  will  occur  to  you..." 
This  advice  was  reiterated  in  condensed  form  when  the  subjects  Indicated 
they  had  completed  the  problem. 

Otner  parts  of  the  instructions  were  devoted  to  interacting  with  the 
computer's  menu  to  1)  Review  the  instructions  of  the  e;^rlment,  2) 
Display  the  problem  on  the  screen  again,  3)  Enter  a  new  act,  4)  Review 
acts  alrea<fy  generated  (to  avoid  duplicaticxi) ,  5}  Get  help  on  any  aspect 
of  the  maiu,  and  6)  Terminate  the  experiment. 

After  the  subjects  had  read  these  instructions,  they  were  given  a  practice 
problem.  The  experimenter  helped  them  during  the  practice  problem  and  did 
not  let  the  subject  move  on  to  the  main  problem  until  it  was  clear  the 
subject  understood  the  experiment.  The  practice  problem  consisted  of 
generating  acts  which  might  be  taken  if  one  ran  out  of  gas  on  the  freeway 
and  had  no  money. 

After  completing  the  practice  problem,  subjects  were  shown  exemplar  acts 
generated  by  a  highly  creative  subject  for  the  sane  problem.  This  was  done 
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to  encxxirage  them  to  think  of  as  many  acts  as  possible. 

Subjects  were  told  there  was  only  one  problem  after  the  practice  problem, 
and  they  were  to  do  the  best  job  they  possibly  could  on  the  single 
prc^lem.  They  were  also  told  they  could  take  as  much  time  as  they  needed 
to  coi^ete  the  main  problem.  'Riese  instructions  were  devel<^»d  on  the 
basis  of  several  pilot  studies,  in  which  it  was  found  working  on  a  single 
problan  leads  to  a  considerable  increase  in  the  number  of  responses 
generated. 

Procedure.  The  subjects  interacted  with  the  computer  by  reading  the 
textual  material  from  a  CBS:  display  and  entering  their  acts  via  the 
keyboard.  Ihe  previously-described  menu  of  comnands  was  used  by  subjects 
to  control  the  computer.  For  exaople,  subjects  generated  acts  by  first, 
hitting  the  "E"  k^  which  signified  they  wanted  to  "Enter"  an  act,  then 
typing  their  act.  Subjects  were  then  asked  to  type  a  short  justification 
for  the  act.  This  was  done  to  aid  the  ei^rimaiters  in  interpreting 
subjects'  reasoning  behind  suggesting  some  of  the  acts.  After  the 
justification  was  entered,  the  coffpiter  returned  to  the  menu.  Mflnai 
subjects  chose  to  end  the  problem,  they  were  asked  to  think  again  of  any 
acts  which  might  successfully  solve  the  problem. 

Subjects  then  generated  as  many  acts  as  they  could  for  either  the  Parking 
or  the  Living  problem.  The  experimenter  was  present  for  the  entire  session 
and  answered  questions  at  any  time. 

Beaulta  JOd  Discussion 

Creatinq  ^  act  fiOCl  ]2y  combining  equivalent  acts.  Subjects  generated  a 
total  of  335  acts  for  the  Parking  problem  and  362  acts  for  the  Living 
problem.  Many  acts  were  equivalent  or  highly  similar,  althou^  the  words 
used  to  describe  the  acts  differed.  To  combine  equivalent  acts,  three 
es^rlmenters  served  as  judges  to  independ&itly  group  acts  which  they  felt 
were  equivalent.  For  exanple,  a  suggestion  to  build  a  six-story  parking 
garage  was  ocxisidered  to  be  equivalent  to  building  a  ten-story  garage.  The 
equivalence  judgnents  were  then  analyzed  by  a  conputer  program.  Acts  which 
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were  classified  as  equivalent  by  all  three  independait  judges  were 
coRibined  into  one  act.  This  procedure  resulted  in  an  act  pool  of  128 
"unique"  acts  for  the  Parking  problem  and  155  "unique"  acts  for  the  Living 
problem. 

hierarchical  orqanizaticgi  of  the  acts.  It  is  not  reasonable  to  e^qpect  a 
subject  to  generate  as  many  acts  as  the  pooled  respcxises  of  30  subjects.  A 
more  realistic  expectation  is  that  a  good  act  generator  should  be  able  to 
retrieve  examples  of  the  major  types  of  acts  from  memory.  Performance  can 
be  evaluated  by  examining  the  responses  of  an  individual  with  respect  to 
some  hierarchiced  structure  created  ty  logical  analysis  of  the  ic3eas 
embcxlied  in  the  unique  a<:±s.  When  the  unique  acts  were  inspected  it  was 
clear  they  cauld  be  organized  into  a  hierarchical  structure.  Certain 
generic  ideas  were  shared  by  many  of  the  ac±s,  and  there  were  major 
variations  of  these  ideas,  as  well  as  minor  variations.  Because  the 
hierarchical  structure  is  tree- like,  we  named  the  generic  icJeas  "limbs", 
major  variations  "branches",  and  minor  variations  "twigs".  By  cxmparing 
the  aca:  tree  generated  by  each  subjec:t  with  the  "group"  tree  created  by 
pooling  the  responses  of  all  30  subjects,  we  can  identify  an  individual's 
omission  of  in^rtant  limbs  and  branches. 

IVro  experimoiters  structured  the  isiicjue  ac±s  generated  for  the  Peurking  and 
Living  problems.  The  results  are  shown  in  /^spmiciices  B  and  C, 
respecxively.  Hie  limbs,  branciies,  and  twigs  in  the  trees  are  demoted  by  a 
decimal  notation.  For  exarofde,  the  notation  2.3.1  for  the  Parking  problem 
refers  to  a  proposal  to  build  a  belcw-ground  parking  stucture.  Hiis  ac:t  is 
a  member  of  generic  limb  2,  "Increase  available  space  for  parking",  and  is 
found  on  brancdi  3,  "Build  parking  structures",  a  branch  it  sheures  with 
another  act,  "A  high-rise  parking  structure"  (2.3.2).  Most  lijitos  contained 
many  branches,  and  some  bremches  had  a  large  number  of  twigs.  Each  twig 
represents  at  least  one  act,  althou^  some  twigs  combine  severed  ac:ts.  Hie 
trees  were  created  with  a  much  more  liberal  (definition  of  e(]uivalenc%  than 
used  in  the  "three  ju<dge"  prcx^edure  to  keep  the  size  of  the  tree  within 
manageable  prc^rtions.  Less  specific  acts  were  associated  with  a  branch 
or  even  a  major  limb,  rather  than  a  twig. 
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An  in^peccion  ot  Appendices  B  and  C  reveals  the  incredible  richness  of  the 
Parking  and  Living  trees  created  from  the  pooled  acts.  This  is  evoi  more 
remarkable  when  it  is  considered  the  trees  provide  lower-bound  estimates 
of  the  "hypothetical”,  acymptotic  trees  created  1:^  including  many  more 
subjects  in  the  pooling  process.  The  Parking  problem  has  seven  major 
limbs,  and  the  Living  problem  has  nine.  Examples  of  major  limbs  in  the 
Parking  prdalem  include  "Use  available  parking  space  more  efficioitly”, 
"Increase  available  space  for  parking",  "Reduce  demand  for  parking",  and 
"Change  parking  priorities".  Other  limbs,  such  as  "Provide  faculty  and 
Staff  with  housing  within  walking  distance”  and  "Frivolous  and  other 
responses",  while  consisting  mostly  of  acts  that  are  patently  unrealistic, 
are  included  because  of  their  Intrinsic  interest.  Obese  limbs  contain 
material  suggestive  of  some  strategies  subjects  use  during  act  retrieval. 
Limb  7,  "Responses  that  combine  two  or  more  options  from  above”,  is  based 
on  the  re^nse  of  one  subject  and  may  have  resulted  from  a 
misunderstanding  on  the  part  of  that  subject.  Obe  instructions  specified 
that  the  teusk  was  to  create  a  menu  of  possible  acts,  and  we  did  not 
specify  that  (mly  act  would  be  picked  from  that  menu.  However,  it  is 
interesting  to  note  only  one  subject  i^cifled  the  possibility  of 
combining  acts. 

The  tree  created  for  the  Living  problem  is  similar  to  the  Parking  problem 
tree  in  many  ways.  Examples  of  major  limbs  included  in  the  tree  for  the 
Living  problem  are  "Live  somewhere  without  paying  rent”,  "Exchange  goods 
or  services  for  mon^  or  place  to  live",  "Obtain  mon^  throu^  other 
sources",  "Ask  someone  for  help",  "Try  to  change  regulations",  "Change 
current  plans",  and  "Prepare  in  advance".  Subjects  often  suggested  acts 
involving  changing  regulations  or  violating  the  constraints  placed  on  the 
problem  by  the  experimenter.  Again,  subjects  suggested  acts  having  a  small 
probability  of  successfully  solving  the  problem.  Those  acts  were  included 
in  two  categories,  "Long  shots”,  and  "Acts  which  will  not  solve  problem". 

nnng«Tiif<»<ny  ^  tree  with  cluster  anaiyaig.  It  could  be  argued  that  the 
structures  for  both  the  Parking  and  the  Living  trees  resulted  from  the 
idiosyncratic  thou^t  processes  of  tl^  individuals  who  created  them.  Other 
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inoivlduais  mi^t  have  produced  different  trees.  If  our  trees  are  to  be 
useful  in  assessing  the  con^eteness  of  act  generation  performance  r  sene 
incdcation  of  their  generality  should  be  available.  Manning  (note  6)  has 
provided  one  such  check  cm  the  generality  of  the  tree  used  to  assess  the 
Parking  problem.  In  an  investigation  of  the  subjective  dimensions  used  by 
subjects  in  thinking  about  acts  for  the  Parking  Problemr  she  performed  a 
complete-linkage  cluster  analysis  on  acts  having  positive  utility.  Ihe 
output  of  this  analysis  is  a  hierarchical  tree,  which  is  given  as  ^f)endix 
D.  Her  tree  includes  only  acts  generated  by  the  subjects  from  the  Parking 
problem  judged  to  be  of  positive  utility.  Dei^ite  this  difference,  there 
are  real  similarities  between  the  etqperimenter-generated  tree  in  Agpendix 
B  and  the  tree  resulting  from  cluster  analysis  in  kpgendix  D.  Her  major 
limbs  1.,  "Do  more  with  University-owned  space"  and  2.,  "Expand  parking  to 
surrounding  areas"  are  similar  to  the  limb  2.,  "Increase  available  ^ce 
for  parking"  in  Appendix  B.  Similarly,  her  limb  3.,  "Use  alternate  forms 
of  transportaticxi"  is  similar  to  branch  3.3  in  Appendix  B,  "Introduce  or 
encourage  alternate  forms  of  transportation".  There  are  differences 
between  the  two  trees,  but  the  differences  are  not  great  enough,  in  our 
opinion,  to  render  the  experimatter-generated  trees  unusable  for  making 
judgnaits  about  the  completeness  of  human  act  generation. 

TlKlIvidllfll  ^  qaaetation  petfonnance.  individuals  in  the  Parking  and 
Living  problems  generated  an  average  of  11.2  and  12.1  acts,  respectively. 
Ihe  number  of  acts  generated  for  the  Parking  and  Living  problems  ranged 
from  2  to  35.  Similar  large  individual  differences  in  performance  were 
observed  by  Manning,  et.  al.  (1980)  in  hypothesis  generation.  Subjects 
generated  an  average  of  3.3  limbs  for  the  Parking  problem  and  4.2  limbs 
for  the  Living  problem.  Subjects  generated  an  average  of  1.92  branches  for 
each  limb  th^  generated  for  the  Parking  problem,  and  1.52  branches  for 
each  limb  generated  for  the  Living  prdalem. 

Ihe  number  of  acts  generated  in  each  limb  or  branch  is  shewn  to  the  IdEt 
of  each  limb  or  branch  in  the  trees  given  in  i^pendices  B,  C,  and  D.  Die 
frequency  of  acts  in  i^pendlx  D  is  Icwer  because  only  positive  utility 
acts  were  included.  Also  included  is  the  probability  of  generating  at 


least  one  act  In  a  limb  or  branch.  For  exampler  the  notation  "14”  followed 
by  "(.37)"  for  limb  1  of  the  Parking  problem  means  that  there  were  14  acts 
in  limb  1  and  11  of  the  30  subjects  (37%)  generated  at  least  one  of  the  14 
acts. 

Inspection  of  these  frequencies  for  the  Parking  problem  tree  yields 
several  conclusions.  First,  while  limb  2,  "Increase  available  ^ce  for 
parking",  and  limb  3,  "Reduce  demand  for  parking",  were  generated  by 
almost  all  subjects  ({^.97  in  each  case) ,  other  major  limbs  such  as  limb  1 
and  limb  4  were  only  generated  by  about  a  third  of  the  subjects.  Limb  1, 
"Use  available  parking  space  more  efficiently",  is  an  important  limb  the 
Universicy  is  currently  inplementing.  Parking  lots  are  being  redesigned  to 
be  more  efficient  using  the  1.1,  1.2,  1.5,  1.6,  and  1.7  branches  shown  on 
the  tree.  Similarly,  rethinking  parking  priorities  (lixob  4)  could 
noticeably  improve  the  parking  situation.  However,  two  out  of  three 
subjects  did  not  think  of  these  (obvious  in  hindsic^t)  limbs. 

Secondly,  while  most  subjects  generated  at  least  oie  act  in  limbs  2  and  3, 
many  other  major  branches  which  one  would  expect  a  large  proportion  of  the 
subjects  to  generate  did  not  show  large  prc^rtions.  For  example, 
providing  incentives  such  as  reserving  parking  places  for  people  vAio 
carpool  (3.4.2)  would  be  an  effective  act  which  would  cost  the  University 
almost  nothing,  yet  this  act  was  only  rarely  suggested  by  subjects. 

A  similar  anedLysis  can  be  made  using  the  tree  constructed  from  cluster 
analysis  for  the  Parking  problem.  Only  30%  of  the  subjects  thou^t  of 
branch  1.2,  "Reorganize  current  parking  areas”,  yet  this  is  the  most 
cost-effective  w£y  to  increase  the  available  parking  given  the  strong 
shift  to  subcompact  cars  in  the  university  comnunity.  Only  37%  of  the 
subjects  thought  of  branch  3.3,  "Pronote  other  forms  of  tran^rtation". 
Bicycle  riding  would  probably  become  much  more  popular  if  a  way  could  be 
found  to  reduce  bicycle  thefts  and  bi<ycle  paths  were  built  to  separate 
bicyclists  from  pedestrians  and  motorists. 

In  the  Living  problem,  90%  of  the  subjects  generated  at  least  one  act  in 
limb  1.0,  "Live  with  someone  without  paying  rent",  and  in  limb  3.0, 
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"Obtain  money  from  other  sources".  Ei^ty  percent  of  the  subjects 
generated  at  least  one  act  in  limb  2. Or  "Exchange  goods  or  services  for 
money  or  a  place  to  live".  However ,  only  27%  of  the  subjects  thou^t  to 
eusk  someone  else  for  housing  suggestions  (limb  4)  and  only  60%  considered 
having  the  friend  modify  his  plans  (limb  6) .  While  most  subjects  goierated 
at  least  one  act  in  limb  Ir  "Live  with  someone  without  paying  rent"r  the 
majority  (xily  considered  having  the  friend  live  with  someone  he  knows 
rather  than  explore  other  sources.  In  limb  2r  "Exchange  goods  or  services 
for  money  or  a  place  to  live"r  63%  of  the  subjects  thou^t  of  selling 
goods  or  services  to  make  mon^r  i^ut  less  than  hedf  considered  exchanging 
his  services  for  living  quarters.  In  limb  3,  "Obtain  money  from  other 
sources"  f  only  a  third  of  the  subjects  suggested  applying  for  financial 
assistance. 

Obviously  there  is  a  subjective  element  in  these  analyses.  Readers  will 
have  to  inflect  the  trees  and  reach  their  own  conclusions.  We  were 
surprised  by  the  defici^cies  in  subjects'  performance  even  though  our 
earlier  research  on  hypothesis  generation  suggested  subjects  would  fail  to 
think  of  inportant  acts. 

Ihe  source  of  these  failures  is  an  interesting  topic  for  speculation.  The 
problems  were  picked  so  we  could  argue  that  most  important  acts  would  be 
"available"  in  roemoryr  but  not  neocessarily  "accessible"  (Tulving  and 
Pearlstoner  1966) .  In  other  words,  we  believe  that  the  requisite 
information  is  available  in  the  memories  of  most  of  our  subjects  dbe  to 
their  experiences  with  personal  transportation,  or  their  experiences  with 
finding  accomnodations  for  themselves  or  their  friends.  Therefore,  we  are 
inclined  to  attribute  most  of  the  difficulty  in  act  retrieval  to  f2dlure 
to  access  information  available  in  memory.  Perhaps  an  eiqiert  would  have 
less  difficulty  accessing  material  used  on  a  daily  basis.  On  the  other 
hand,  we  believe  that  important  decisions  are  not  necessarily  made  by 
experts  who  make  similar  decisions  on  a  daily  basis.  Often  "generalists" 
such  as  politicians,  military  officers,  and  lawyers  are  found  in  decision 
making  roles.  Do  they  have  difficulties  in  accessing  information  similar 
to  our  subjects'  difficulties?  We  suspect  so. 
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We  do  not  believe  many  of  the  failures  to  generate  important  limbs  and 
branches  are  due  to  a  hyper-critical  attitude  on  the  part  of  our  subjects 
towards  the  acts  they  retrieved  from  manory.  As  can  be  seen  from  an 
inspectiixx  of  the  trees,  many  responses  are  counter-productive  or  useless, 
suggesting  the  subjects  were  trying  to  follow  instructions  by  responding 
with  everything  that  came  to  mind.  Why  would  they  make  frivolous  responses 
rather  than  suggest  acts  which  might  have  been  effective? 

we  cannot  preclude  a  motivational  explanation  for  some  failures  to  think 
of  important  limbs  and  branches,  we  went  to  considerable  pains  to  design 
instructions  and  procedures  for  our  ei^riments  to  motivate  the  subjects. 
Most  subjects  ^nt  about  an  hour  in  the  experiment,  which  was  restricted 
to  a  single  problem.  Nai^  subjects  found  the  e^qseriment  intrinsically 
interesting  and  treated  it  as  a  contest  or  a  game.  Nevertheless,  seme 
retrieval  failures  probably  were  due  to  insufficient  motivation. 

Our  conclusion  from  inspecting  these  data  is  that  the  tree  produced  by  the 
individual  subject  often  lacks  important  limbs  and  branches.  We  presented 
almost  all  the  data  used  in  reaching  this  conclusion  so  the  readers  chn 
reach  their  own  conclusions.  If  our  conclusioi  is  accepted,  the  next 
logical  step  is  to  assess  the  cost  of  failing  to  generate  a  complete  tree. 


Ebqperiinent  2. 

The  seoond  major  question  of  opportuiity  loss  can  be  addressed  by  finding 
the  utility  of  all  unique  acts  in  the  "pooled-acts"  tree.  Once  the 
utilities  of  various  branches,  limbs,  and  twigs  are  known,  the  tree 
produced  by  each  subject  can  be  compared  to  the  pooled-acts  tree,  and 
opportunity  loss  c»n  be  c:alcxilated.  If  subjects  do  not  generate  a  cxmplete 
tree,  but  generate  all  important  limbs  and  branches,  opporttxiity  loss  will 
be  negligible.  If,  however,  important  limbs  and  branches  are  omitted, 
opportunity  loss  will  be  large. 

Figuratively,  the  first  ejqjeriment  examined  how  well  subjects  can  sketch 
the  pooled  acts  tree,  and  the  remaining  experiments  examined  whether  the 


brandies  th^  sketch  out  are  "heavy  with  fruit".  An  examination  of  the 
oonpleteness  of  the  structure  is  inccmpLete  without  an  assessment  of  the 
consequenoes  of  failing  to  generate  branches  and  limbs.  Whereas  the 
previous  analysis  relied  heavily  on  such  words  as  "important"  or  "useful" 
and  invited  the  reader  to  agree  or  disagree  with  this  judgment,  an 
<%)portunity  loss  calculation  injects  an  elanent  of  objectivity  into  the 
analysis. 

Our  primary  difficulty  in  performing  an  opportunity  loss  calculation  was 
that  the  "pooled-^act"  trees  were  nnich  larger  than  we  anticipated.  We  had 
planned  to  use  a  version  of  Multi-Attribute  Utility  scaling  to  estimate 
the  utility  of  each  act,  but  the  large  number  of  unique  acts  made  this 
inp^ssible.  Instead,  we  were  fc»:ced  to  settle  for  first,  screening  the 
unique  acts  to  identify  feasible  acts  of  positive  utility,  and  second, 
evaluating  the  utility  of  the  screeied  acts  by  using  a  gl<±al  utility 
estimation  technique. 

Consequently,  we  performed  e^qieriment  2  to  provide  a  preliminary  screening 
of  the  acts.  The  technique  used  was  to  ask  subjects  to  identify  acts 
having  positive  utility.  E]q»rlment  3  provided  global  utility  estimates 
for  a(±s  judged  to  be  of  positive  utility  by  most  of  the  subjects  in  the 
preliminary  screening. 

Method 

Subjects  were  60  introductory  psychology  students  from  the 
University  of  Oklahoma  idx)  participated  in  the  experiment  for  csourse 
credit.  Thirty  subjects  saw  the  Parking  problem  and  30  subjects  saw  the 
Living  problem. 

PronwAiro.  The  ejq^eriment  was  cxnputer-controlled.  Subjects  first  read  the 
practice  problem  seen  fy  subjects  in  the  act  generation  experiment.  Ihe 
prac!tl(%  problem  dealt  with  acts  to  be  taken  if  the  subject  ran  out  of  gas 
and  had  no  money.  Subjects  were  shewn  4  sample  acts  and  were  asked  to 
respond  "yes"  or  "no"  to  the  question  of  whether  taking  each  iqwcified  act 
would  leave  them  better  off.  A  response  to  the  question  was  made  by  typing 
either  "Y"  or  "M"  on  the  keyboard.  After  each  response,  subjects  were 
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given  an  opportunity  to  look  at  the  problem  again  or  to  move  on  to  the 
next  act. 

Following  the  practice  problem^  subjects  read  either  the  Parking  or  the 
Living  problem.  Subjects  were  told  their  task  was  to  decide  whether  taking 
the  specified  acts  would  leave  the  involved  parties  better  off  than  they 
currently  were.  Subjects  were  also  encouraged  to  wei^  the  costs  of  taking 
any  act  against  its  potential  boiefits  when  deciding  whether  the  act  good. 
Subjects  again  evaluated  acts  by  typing  either  "Y"  or  ”N”  on  the  ke^x>ard. 
Subjects  who  saw  the  Barking  problem  evaluated  128  acts  and  subjects  who 
saw  the  Living  problem  evaluated  155  acts. 

iteHuit-R  The  nuttber  of  subjects  who  thou^t  an  act  was  of 

positive  utility  was  determined.  If  the  majority  of  subjects  in  the 
preliminary  screening  experim^t  rated  an  act  as  having  positive  utility^ 
it  was  retained  for  use  in  the  utility  experiment.  Fifty-seven  acts  were 
classified  as  being  of  positive  utility  in  the  Parking  problemr  and  67  in 
the  Living  problem. 

The  procedure  used  for  preliminary  screening  was  somewhat  cruder  but  it 
should  be  remembered  that  <^)portunity  loss  is  calculated  by  finding  the 
difference  between  the  IsesL  act  <xi  the  "pooled  acts"  tree  and  the  Jasst  act 
cn  a  subject's  tree.  Consequently r  the  utility  of  acts  having  negative 
utility  need  not  be  knownr  as  the  influoice  of  negative  utility  acts  on 
the  opportunity  loss  calculation  is  negligible  (except  in  pathological 
cases) . 


firoeglBiRnt.  1 

TRBhnlgmn  tOL  nBnpminq  act  omeratlmi  performance 

• 

Many  of  the  acts  in  the  Parking  and  Living  problems  are  not 
mutually-exclusive.  Thusr  it  should  be  possible  to  create  a  "portfolio" 
(Cozzolinor  1974)  of  acts  to  address  the  problem.  For  exampler  parking  at 
the  University  of  Oklahoma  could  be  improved  by  redesigning  existing 
parking  lotSr  building  new  lotSr  inproving  mass  transitr  etc.  On  the  other 
handr  the  Canadian  student  can  only  live  in  one  place  at  a  timer  althou^ 


other  actsr  such  as  getting  more  moneyr  could  be  combined  with  acts 
suggesting  where  be  might  live.  Therefore,  a  subject's  performance  should 
be  evaluated  in  terms  of  both  the  best  act  generated  and  the  number  of 
hi^  utility  acts  generated,  because  many  acts  could  be  takei 
simultaneously . 

Portfolio  analysis  is  a  formal  technique  to  idaitify  the  best  mixture  of 
acts  to  be  taken.  Unfortunately,  the  problems  we  are  considering  are  not 
suitable  candidates  for  such  an  analysis  because  some  acts  are  mutually 
exclusive.  Instead,  we  develc^d  an  analysis  which  embodies  seme  of  the 
ideas  of  portfolio  analysis  and  addresses  other  dimensions  of  performance 
as  well. 

Sappoae  a  subject  produces  an  act  tree  consisting  of  limbs,  branches,  and 
twigs.  Each  act  twig  has  an  associated  utility  value  obtained  from  a 
utility  scaling  procedure.  Also  imagine  a  comparable  tree  created  by 
pooling  the  acts  of  the  entire  grot^  of  30  subjects.  The  "group”  tree  will 
have  many  more  branches  and  twigs,  but  if  subjects  perform  well  their 
"individual"  trees  should  resemble  a  pruned  "groqp”  tree  containing  only 
the  best  limbs,  branches  and  twigs. 

A  tree  can  be  pruned  at  various  levels.  TWigs  can  be  pruied  from  a  branch 
until  only  the  highest-utility  twig  remains.  After  twigs  eu:e  thus  pruned 
from  each  branch,  branches  retaining  the  best  twigs  can  be  pruned  from  a 
limb  until  only  the  highest-utility  branch  remains.  Finally,  limbs 
retaining  (xily  their  best  branch  and  twig  can  be  prvned  until  only  the 
hi^st-utility  limb  remains. 

If  both  the  "group"  and  "individual"  trees  were  pruned  to  their  single 
best  limb,  branch  and  twig,  the  act  having  maximum  utility  for  each  tree 
would  be  idmitified.  The  difference  between  the  maxiimm  utility  act  for 
the  "groip"  tree  and  an  "individual"  tree  would  be  the  "<^portunity  loss" 
resulting  from  the  subject's  failure  to  generate  the  best  possible  act. 

If,  however,  for  both  trees  priming  stops  after  each  branch  is  prined  so 
only  the  best  twig  (in  a  roaxiimmi  utility  sense)  remains,  then  a  pruned 
tree  would  consist  of  limbs  and  branches  containing  only  the  best  twig 


available  for  that  limb  and  branch.  If  the  analysis  is  extended  to  these 
additional  limbs  and  branches,  an  idea  of  the  variety  and  completeness  of 
the  subject's  tree  structure  can  be  obtained.  A  ccRqparision  between  the 
best  limb  and  branch  of  the  two  can  still  be  made  to  calculate  opportunity 
loss.  The  additional  limbs  and  branches  can  be  examined  to  assess  the 
completeness  of  the  "individual"  tree.  We  have  mned  this  analysis  "limbs 
and  branches". 

Onoe  the  tree  has  been  pruned  to  this  extent,  it  is  a  simple  matter  to 
prune  it  further,  so  each  limb  retains  only  its  maximum  utility  branch  and 
twig.  The  analysis  performed  on  this  b/pe  of  tree  is  named  "limbs  only". 
The  "limbs  and  branches"  analysis  said  the  "limbs  only"  analysis  provide 
somewnat  complementary  information.  If  a  "limbs  and  branches"  analysis  is 
performed,  a  oomparlsion  of  the  groip  tree  with  the  Individual  tree  is 
ess^tially  an  examination  of  how  well  a  subject  generates  limbs  and 
branches.  If,  on  the  other  hand,  a  "limbs  only  analysis  is  performed,  the 
emphasis  is  on  how  well  a  subject  generates  various  limbs. 

Suppoae  that  the  various  versions  of  the  group  and  individual  trees 
outlined  above  are  created.  One  way  to  suranarize  subjects'  for  limbs  only 
performance  is  to  create  a  cumulative  utility  function  which  fiUDB  the 
utilities  of  the  limbs  in  order  of  decreasing  utill^.  This  is  done  by 
first  taking  the  hic^st  utility  limb,  adding  this  utility  to  the 
second-highest  utility  limb,  etc.  A  similar  analysis  may  be  performed  on 
the  trees  consisting  of  limbs  and  branches.  The  cumulative  utility 
function  resulting  from  this  process  increases  monotonically,  and 
eventually  is  ayaeptotic.  If  the  cumulative  utility  functions  are 
calculated  and  plotted  for  the  appropriate  versions  of  the  group  and 
individual  trees,  a  sumsHury  measure  is  created  which  captures  both  a 
subject's  ability  to  generate  the  "best"  act,  and  the  ability  to  generate 
a  variety  of  other  good  acts.  The  difference  betwe«)  the  first  points  on 
two  of  these  functions  is  "opportunity  loss",  and  the  differeioes  between 
their  rcBSkining  points  can  be  used  to  assess  the  "oomptleteness"  of 
performance. 
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Utility  patimatifvn  technique.  Ttie  utility  estimation  technique  enjoyed 
was  direct  utility  estimation.  Subjects  were  given  two  "anchor”  acts,  one 
identified  as  having  a  value  of  0,  the  other  a  value  of  100.  Ihey  then 
made  global  utility  estimates  for  all  other  acts  by  oonparing  each  act 
with  the  two  anchor  acts.  If  subjects  believed  an  act  had  a  hi^er  utility 
thcJi  the  act  identified  as  having  a  utility  of  100,  they  were  free  to 

assign  it  a  larger  number.  If  they  believed  an  act  had  a  lower  utility 

than  the  0  utility  anchor,  they  could  assign  it  a  negative  number.  The 
acts  which  served  as  anchors  were  selected  using  the  results  of  e3q)erlmait 
2.  For  the  Parking  problem,  the  act  "Inqprofve  the  trolley  and  Canpis  Area 
Rapid  Transit  System...”  was  used  as  the  upper  anchor  and  was  assigned  a 
value  of  100  because  more  subjects  (29  out  of  30)  classified  the  act  as 
being  of  positive  utility  than  ai^  other  act  in  experimeit  2.  A  utility  of 
zero  was  assigned  to  the  act  of  doing  nothing  about  the  Parking  problem. 
The  Living  problem  differed  from  the  Parking  problem  in  that  the  act  ”do 

nothing”  was  not  viable,  as  the  student  needed  a  place  to  live. 

Gcxisequently,  we  picked  an  act  for  a  lower  anchor  which  few  subjects  (3 
out  of  30)  in  experiment  2  felt  was  of  positive  utility.  This  act  was  ”... 
hide  in  the  library  and  sleep  there  at  night...”.  VAiile  some  studoits 
resort  to  similar  tactics,  this  is  obviously  a  poor  solution  to  the 
prdslem.  The  act  chosen  to  be  the  upper  anchor  involved  appealing  to 
relatives  for  help  and  was  picked  because  all  subjects  thought  it  was  of 
positive  utility. 

The  two  utility  scales  defined  for  the  Parking  and  the  Living  Problems  are 
not  ooBinensurate,  because  thq^  have  different-valued  intervals. 
Consequently,  cooparisions  betweoi  the  Parking  and  Living  problems  cannot 
be  made  directly,  because  the  scales  are  technically  Interval  scales  and 
are  only  unique  up  to  a  linear  transformation.  This  issue  will  be 
discussed  in  more  detail  later. 

Subjects.  TVrenty  subjects  were  used  for  each  problem.  They  were  recuited 
from  the  same  population  as  previous  subjects. 
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instructiona.  Hue  instructions  to  tl^  subjects  were  similar  to  those  used 
in  experiment  2.  Subjects  were  told  to  rate  acts  generated  by  other 
subjects  by  making  numerical  utility  estimates.  They  were  instructed  that 
their  global  estimates  should  incorporate  the  costs  of  impLeroenting  a 
particular  act  and  possible  outcomes  resulting  from  that  act. 

Procedure.  The  procedure  for  the  practice  problem  was  the  seme  as  in 
experiment  2,  except  subjects  made  direct  utility  estimates.  The  computer 
screen  di^leyed  three  acts.  The  two  anchor  acts  were  alw^s  present  at 
the  top  and  bottom  of  the  screen,  and  were  labeled  0  and  100.  The  act  to 
be  judged  was  displayed  in  the  middle  of  the  screen  between  the  two 
anchors.  Subjects  entered  a  utility  value  into  the  computer  terminal  which 
represented  their  evaluation  of  the  utility  of  the  act  to  be  rated  as 
compared  with  the  two  anchor  acts.  Hie  computer  then  advanced  to  the  next 
act  to  be  rated. 

When  the  subjects  finished  the  practice  problem,  they  read  the  text  of 
either  the  Parking  problem  or  the  Living  problem.  They  made  direct  utility 
estimates  for  57  acts  from  the  Parking  problem,  or  67  acts  from  the  Living 
problem.  To  insure  subjects  would  be  conscientious,  th^  were  required  to 
justify  the  magnitude  of  their  response  for  six  randomly  chosen  acts.  Hie 
subjects  did  not  know  for  which  acts  they  would  be  required  to  justify 
utility  estimates  until  after  they  entered  the  estimate. 

Beaultfi  flod  discussion 

Hie  subjects'  utility  estimates  were  examined  ty  plotting  histograms  of 
the  distribution  of  the  estimates  for  each  problem.  Almost  all  histogrems 
were  unimodal,  but  there  was  considerable  variation  among  the  subjects' 
estimates.  Hie  median  utility  estimate  was  used  as  a  measure  of  central 
toidency  to  reduce  the  influoice  of  outlying  scores. 

Hie  "Tiniift  only"  arvl  "Hmha  and  branches"  analyBAs.  A  conqputer  program 
performed  the  "limbs  only"  and  the  "limbs  and  branches"  analyses  described 
above.  It  also  perfcmmed  a  similar  analysis  on  unprizied  trees,  but  the 
results  of  this  analysis  were  so  similar  to  those  of  the  "limbs  and 
branches"  analysis  that  they  will  not  be  reported.  The  input  to  the 
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program  was  the  classificaticn  of  acts  created  by  the  e^qperlmenters  and 
the  median  utility  estimates  detained  fron  experiment  3.  The  outyut  of  the 
program  was  a  cumulative  utility  function  for  the  group  tree  consisting  of 
pooled  acts  generated  by  all  subjects.  This  function  shows  the  emulative 
utility  for  the  total  number  of  limbs  for  the  "limbs  (xily"  analysis^  and 
the  ten  highest  utility  limbs  and  branches  for  the  "limbs  and  branches" 
analysis.  In  each  case  the  first  point  in  the  function  is  the  maximm 
utility  actf  the  second  point  is  the  second-ranked  act,  etc.  Table  1  shows 
the  data  used  in  the  groip  "limbs  and  branches"  analysis  for  the  Parking 
problem.  The  frequencies  of  generating  acts  illustrate  individual 
performance  and  were  not  a  part  of  the  groc^  analysis. 


Table  1 

The  best  10  limbs  and  branches  from  the  "group"  Parking  problan  tree 


Utility 


Nmber  of 

subjects 

generating 


Action 


100 

77.5 
60 
60 

57.5 
55 
55 
51 
50 
50 


2 

19 

2 

1 

1 

3 

2 

3 

1 

3 


improve  the  trolley  and  CART  tystems. 

Build  a  hi^rise  parking  lot. 

Put  more  anall  car  parking  spaces  in  studoit  lots. 
Have  aiployees  park  at  Lloyd  Noble,  use  trolley. 
Build  lots  around  Norman,  shuttle  in  students. 
Reduce  the  price  of  parking  stickers  for  carpools. 
Advertise  advantages  of  riding  bike,  motorcycle. 
Have  more  selection  of  afternoon,  evening  courses. 
Use  some  CXJ  service  v^icle  parking  for  studmits. 
Use  extra  areas  around  fraternities  for  overflow. 


616  30  possible 
sm  of  for  each  act 
utilities 


The  acts  shown  in  table  1  are  the  10  best  acts  from  the  grorp  tree  for  the 
Parking  problem.  The  emulative  utility  functim  was  obtained  from  the 
column  of  utilities  shown  in  the  table.  Also  shown  is  the  number  of 


subjects  generating  each  act.  The  individued  analyses  were  similar  to 
this,  with  one  important  exception.  The  ten  best  acts  for  an  individial 
were  not  necessarily  those  shown  in  table  1,  but  were  the  ten  best  acts 
generated  by  the  individual.  An  inspection  of  the  number  of  subjects 
generating  the  acts  used  in  the  group  analysis  shews  the  typical  subject's 
ten  best  acts  included  some  acts  from  the  group  analysis,  but  edso 
included  many  lower-utility  acts  not  shown  in  table  1.  Therefore,  the 
individual  amalyses  were  similar  in  principle  to  the  group  analysis,  but 
were  based  on  individual  performance. 

Ihe  results  of  the  "limbs  and  branches”  and  the  "limbs  only"  emalyses  are 
shown  in  figures  1  through  4  for  the  P2u:lcing  and  Living  prohlans.  Each 
figure  shows  the  "group"  cumulative  utility  functiai,  the  "mean"  function 
obtained  by  averaging  the  cumulative  utility  functions  for  each  subject, 
and  the  "best"  function  obtained  from  the  best  subject  in  the  analysis. 
The  difference  beween  the  first  point  of  the  "grotp"  function  and  the 
first  point  of  either  of  the  other  functions  is  the  opportunity  loss  due 
to  failure  to  generate  the  highest  utility  act. 

Opportunity  loqp  results.  The  opportunity  loss  due  to  failure  to  generate 
the  best  act  was  29  utiles  (100-71)  for  the  Parking  problem  and  18.15 
utiles  for  the  Living  problem  (100-81.85).  One  way  of  thinking  about  the 
meaning  of  the  opportunity  loss  figures  is  to  imagine  the  utility  distance 
between  the  average  subject's  performance  and  the  ipper  and  lower  anchors 
used  in  experiment  2.  The  utility  of  the  best  act  for  the  average  subject 
in  the  Parking  problem  was  71.  This  utility  value  is  71  perceit  of  the 
distance  from  the  lower  anchor,  and  29  percent  of  the  distance  to  the 
vpper  anchor.  The  utility  of  the  best  act  for  the  average  subject  in  the 
Living  pcoblem  was  81.85.  This  utility  value  is  81.85  percent  of  the 
distance  from  the  lower  anchor  and  18.15  percent  of  the  distance  to  the 
ipper  anchor.  It  should  be  recalled  that  these  opportimity  loss  figures 
are  lower-bouid  estimates.  The  actual  opportunity  loss  values  may  be 
somewnat  hi^r. 

One  cannot  conclude,  however,  that  the  performance  of  subje<±s  in  the 
Living  problem  was  superior  to  the  performance  of  subjects  in  the  Parking 
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FIGURE  1 .  CUMULATIVE  UTILITY  FUNCTIONS  FOR 
"LIMBS  AND  BRANCHES"  ANALYSIS 
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problem  because  the  two  utility  scales  are  not  comnensurate.  Utility 
scales  are  unique  up  to  a  linear  transformation.  No  data  are  available  to 
translate  the  utility  units  of  the  Parking  scale  into  the  units  of  the 
Living  scale.  Our  impression  is  that  the  opportunity  loss  values  aze 
roughly  OGnparable  for  the  two  problems.  It  will  be  recalled  that  the  act 
serving  as  the  lower  anchor  in  the  Parking  problem  was  to  do  nothingr 
whereas  in  the  Living  problem  it  was  to  sleep  in  the  Library.  Therefore, 
the  anchors  in  the  Living  problem  may  enccnpass  a  greater  range  of  utility 
than  the  anchors  for  the  Parking  Problem,  suggesting  each  utility  point  in 
the  Living  problem  scale  has  greater  value  than  each  point  in  the  Parking 
problem  scale.  Therefore,  the  c^portunity  loss  of  29  utiles  for  the 
Parking  problem  may  represent  about  the  same  loss  as  the  18.15  utile  lose 
for  the  Living  problem. 

Old  subjects  generate  many  hlah-valued  limbs?  The  question  of  the 
ccmpleteness  of  each  subject's  tree  is  pertinent  to  the  topic  of  decision 
prdslem  structuring.  Decision  analysis  employs  many  powerful  and 
sophisticated  techniques  to  decide  if  an  act  should  be  included  in  a 
decision  analysis.  Obviously,  if  a  decision  maker  fails  to  generate  the 
act,  the  benefits  of  subsequent  decision  analysis  are  lost.  Decision 
analysis  depends  on  the  quality  of  the  decision  maker's  work  when 
generating  elenents  of  the  structure.  This  issue  can  be  addressed  by 
examining  the  remainder  of  the  cumulative  utility  functions  shown  in 
figures  1  through  4.  The  difference  in  utility  between  the  first  and  last 
points  on  each  cumulative  utility  function  is  related  to  the  completeness 
of  the  associated  tree. 

Table  2  displeys  the  differences  between  the  "group”  and  "mean"  cumulative 
utility  functions.  For  example,  in  the  "limbs  and  branches"  analysis  for 
the  Parking  prc^em,  the  difference  between  the  first  and  last  points  of 
the  "mean"  cumulative  utility  function  was  97.6  (168.6-71).  The  difference 
between  the  first  and  last  points  of  the  "group"  cumulative  utility 
function  was  516  (616-100).  Another  colunn  shows  the  ratios  (expressed  as 
percents)  of  the  utility  differences  for  the  "group"  function  to  the 
utility  differences  for  the  "mean”  function  for  each  problem. 
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Table  2 

Utility  Dlfferencee  from  the  emulative  Utility  functions 


Parking  Problem 


Living  Problem 


"Hean" 

"Group" 

Per  omit 

"Mean"  "Group" 

Percent 

"limbs  and 

branches 

analysis" 

97.6 

516 

528% 

201.9 

750 

371% 

"limbs  only 
analysis*^ 

52.2 

187 

358% 

129 

463.5 

359% 

Table  2  reveals  an  interesting  result.  Subjects  are  unable  to  generate  a 
complete  assortment  of  hlg^  utility  acts  such  as  those  a  Decision  Analyst 
ml^t  hope  to  elicit  from  a  client.  Subjects  do  not  generate  many  hi^ 
utility  limbs  and  branches.  When  the  analysis  is  confined  to  limbSr  a 
similar  picture  emerges.  In  fact,  after  inspecting  the  data  from  which  the 
opportunity  loss  calculatlcms  were  derived,  we  have  develc^ad  a  rather 
unflattering  picture  of  human  act  generation  capabilities. 

First,  many  acts  occurring  to  subjects  are  of  zero  or  negative  utility. 
Subjects  in  the  Parking  problem  generated  an  average  of  11.2  acts.  Of 
these  11.2  acts,  4.4  or  39%,  (m  the  average,  were  judged  to  be  of  positive 
utility.  Only  a  few  of  the  4.4  positive  utility  acts  generated  by  the 
average  subject  were  of  hig^  utility.  Hie  results  from  the  Living  problem 
were  similar.  Itie  average  subject  generated  12.1  acts.  Of  these  12.1  acts, 
6.5,  or  54%,  on  the  average  were  of  positive  utility.  Subjects  in  the 
Living  problem  tended  to  generate  more  high-utility  acts,  and  tended  to 
generate  acts  of  higher  utility.  Table  3  sunmarizes  the  data  on  which 
these  conclusions  were  based.  It  gives  the  mean  nmber  of  acts  in  each 
utility  class  interval  for  the  two  problems  for  indivdual  subjects.  Also 
given  are  the  nmber  of  acts  from  the  group  tree  in  each  utility  class 
interval. 


Table  3  presents  the  relationship  between  utility  of  acts  and  the 
frequency  with  which  the  acts  were  generated  by  individual  subjects.  A 
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"perfect"  act  generator  should  perform  at  least  eis  well  as  the  groip  tree, 
but  we  do  not  expect  our  subjects  to  approach  perfection,  the  cuniu].atlve 
utility  functions  address  subject  performance  in  a  more  realistic  way. 
Table  3  shows,  however,  subjects  generate  only  a  few  viable  acts,  far 
fewer  than  would  be  desirable  from  the  point  of  view  of  decision  analysis. 


Table  3 


The  Utility  Distribution  of  Acts  Generated  by  Subjects 
Compared  to  the  Group  Tree 


Utility  values 
(Midpoints  of 
interval) 

0* 

10 


I 

100 


Parking  Problem 
individual  group 


Living  Problem 
individual  group 


}:19 


1 


*  This  class  interval  contains  acts  eliminated  in 
experiinmt  2.  Many  of  these  acts  would  have  tad 
negative  utili^  estimates  tad  th^  been  included 
in  experimtat  3. 

**  These  entries  are  the  mean  number  of  acts  having 
utilities  within  the  specified  range. 


A  check  sa  generality  tbeSfi  results  yia  £lUfitfiC  analysis.  The 
"limbs"  and  "limbs  and  branches"  analyses  described  above  were  repeated 
for  the  tree  constructed  for  the  Parking  problem  using  cluster  analysis. 
Graphs  of  the  cunulative  utility  functions  for  these  analyses  are  shown  in 
appendices  E  and  F.  Similar  ocxxausions  were  reaohed  from  the  analyses. 
The  qpportunity  loss  calculation  was  unchanged  in  this  analysis,  as  it  is 
not  atfecced  by  the  structure  of  the  tree  used  in  the  analysis.  Utility 
differences  were  calculated  using  the  tree  derived  from  the  cluster 
analysis  and  are  reported  with  the  correcfponding  results  from  table  2  in 
peurentheses  following  each  result.  For  the  "limbs  and  branches"  analysis, 
the  "group"  function  tad  a  utility  difference  of  539  (516)  utiles.  The 
"mean"  individual  function  showed  a  utility  difference  of  119.5  (97.6) 
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utiles.  A  ratio  of  451%  (528%)  was  found  between  the  group  and  mean 
individual  performance.  In  the  "limbs  oxily"  analysis,  the  group  value 
showed  a  210  (187)  utile  difference  and  the  corresponding  "mean" 
individual  function  showed  a  difference  of  87.5  (52.2)  utiles.  Aie  ratio 
between  grocp  and  "mean"  individual  performance  was  240%  (358%) . 

While  changing  the  structure  of  the  tree  seems  to  slightly  affect  the 
results,  the  same  general  conclusions  can  be  made  with  regard  to  the 
assessment  of  the  completeness  of  subjects'  act  generation  performance. 
Subjects  do  not  generate  mary  high  utility  acts.  In  fact,  their  abilii^  to 
generate  possible  acts  in  a  decision  situation  can  be  fairly  characterized 
as  "in^verished". 


Conclusions 

EQQlinai  trees,  utility  estlmationi  ao  adysnCS  in  methodalofflliu 

The  methodology  used  in  these  experiments  represents  an  advance  over 
earlier  techniques  used  to  study  act  generati(xi.  Lorge,  Fox,  Davitz  and 
Brenner  (l!68)  surveyed  a  number  of  studies  contrasting  individual  and 
group  performance,  some  of  which  were  concerned  with  act  generation.  These 
studies  typically  evaluated  performance  counting  the  number  of 
responses  subjects  made.  The  more  recent  stuc^  of  Pitz,  et.  al.  (1980) 
used  a  similar  technique. 

The  pooling  technique,  the  ocxistruction  of  hierarchical  act  trees,  and  the 
estimation  of  utility  of  acts  all  represent,  we  believe,  advances  in 
methodology  ^ch  allow  the  researcher  to  address  issues  of  quality  as 
well  as  While  these  methods  have  problems,  such  as  the  effort 

required  to  implement  them,  they  may  prove  useful  in  other  contexts.  For 
exaspO-e,  the  efficacy  of  various  structuring  techniques  used  in  decision 
analysis  could  be  examined.  These  methods  should  also  prove  generally 
useful  for  the  stuc^  of  predecisicxi  processes.  The  availability  of  a 
hierarchical  act  tree  constructed  pooling  subjects'  acts  should 
facilitate  the  study  of  the  cognitive  processes,  such  as  memory  search 
strategies,  involved  in  predecision  processes.  Th^  should  also  facilitate 
the  stu<^  of  rules  a  decision  maker  uses  to  determine  whether  a  potential 


act  should  be  ccmsidered  further. 

Generalizing  these  results  SOntextS 

Problem  solving  tafiks.  ihe  problems  employed  in  this  stufy  are 
representative  of  an  important  class  of  problems  in  which  the  goal  is 
fairly  well  defined,  but  the  steps  needed  to  reach  the  goal  are  not 
specified  (Taylor,  1974).  Most  problem-solving  researdi  has  beoi  confined 
to  problems  in  which  the  goal  and  the  steps  taken  to  reach  the  goed  are 
well  defined.  For  ex2Dnple,  in  chess  (de  Groot,  1965) ,  potential  acts 
involve  a  limited  set  of  moves,  and  the  problem  confronting  the  ptL«^r  is 
to  arrive  at  the  best  sequence  of  acts.  Problems  in  wldch  the  possible 
acts  are  unspecified  are  more  caimon,  and  usually  are  more  important.  We 
suspect  the  lack  of  attention  to  problems  in  which  possible  acts  are  not 
specified  is  due  to  their  apparent  intractability.  Our  research  suggests 
that  the  intractability  is  more  apparent  than  real.  In  any  event,  the 
results  reported  here  would  seem  to  be  relevant  to  problem  solving,  and 
should  not  be  considered  relevant  to  decision  theory  issues  alone. 

Expertise  revisited.  Earlier  a  distinction  was  drawn  between  an  expert  who 
performs  a  task  on  a  daily  basis  and  has  possible  acts  readily  accessible 
in  memory,  and  the  "generalist"  who  may  not  be  £dole  to  easily  access  acts 
in  memory.  In  our  past  research  on  hypothesis  generation  (Nehle,  note  5; 
Gettys,  et.  al. ,  note  4) ,  experts  and  nonexperts  attenpted  to  generate 
possible  reascxis  why  an  autonobile  malfunctioned.  The  results  from  both 
populations  were  similar,  suggesting  experts  and  nonexperts  have  similar 
deficiencies  in  hypotheses  generation  due  to  the  operation  of  certain 
cognicive  mechanisms.  It  was  not  claimed  that  expertise  was  of  no  value; 
many  tasks  cannot  be  performed  unless  the  requisite  information  is  in 
memory.  It  is  too  early  to  knew  whether  similar  conclusions  can  be  drawn 
for  act  generation.  However,  the  magnitude  of  the  results  reported  here 
suggests  the  variable  of  expertise  should  be  investigated  in  act 
generation. 


£110190  oct  generation  perfonaance  nRfyRnpdi 

In  sumnary,  these  experiments  address  two  major  questions,  one  dealing 
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with  the  conpleteness  of  act  generation  performance,  and  the  other  dealing 
with  the  cost  of  failure  to  generate  ia^rtant  acts.  Subjects  appeared  to 
be  inpoverished  act  generators,  suffering  both  in  the  and  the 

of  acts  generated. 
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A{;pendix  A 
A  PLACE  TO  LIVE 

linagine  ttiat  your  friend  is  a  new  OU  student  from  Canada.  He  was  able  to 
earn  and  borrow  enou^  to  move  to  Norman  and  pay  his  tuition  and  fees  but 
he  does  not  have  ^ou^  money  to  rent  an  apartment  or  live  in  the  dorms. 
In  fact,  his  monthly  check  from  his  parents  will  barely  pay  for  his  meals. 
Somehow  he  has  to  make  arrangments  for  a  place  to  live  for  the  rest  of  the 
semester. 

Ihe  US  Inmigration  Service  prohibits  foreign  students  from  working,  so 
suggesting  that  your  friend  get  a  job  to  earn  mon^  to  rent  an  apartment 
would  not  solve  the  problem.  Also,  your  frimid  has  exhausted  his  credit  at 
Financial  Aids,  and  cannot  get  loans  from  any  other  source.  Your  frigid 
will  not  do  anything  illegal  that  might  result  in  his  being  deported. 

Your  task  is  to  suggest  as  many  possible  solutions  for  his  problem  as  you 
can.  Ihese  solutions  need  not  be  perfect  or  ideal,  in  fact  solutions  that 
have  definite  drawbacks  are  entirely  acceptable  as  long  as  they  have  the 
possiblility  of  solving  his  problem  for  the  rest  of  the  semester.  Your 
solutions  to  this  problem  should  not  be  minor  variations  of  the  same 
solution. 

It  is  in^rtant  that  you  enter  all  new  colons  which  occur  to  you.  Ihis  is 
similar  to  'thinking  out  loud.'  We  do  not  want  you  to  censor  your  options 
and  only  alter  the  ones  which  you  think  are  particularly  good.  Put  down 
all  options  which  occur  to  you,  good  or  bad.  Your  score  in  this  experiment 
is  determined  by  the  number  of  c^icxis  you  generate  not  by  hew  good  they 
are. 


APPENDIX  B 

PCS  SIMJCIURE  FCR  PARKING  PROBLEM 
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Baaed  on  all  "logically  distinct"  acts,  PARKING  experiment 


14  *  1. 

(.37)  ** 

Use  aval: 

2 

(.07) 

1.1 

3 

(.07) 

1.2 

(.03) 

1.3 

2 

(.07) 

1.4 

2 

(.07) 

1.5 

3 

(.10) 

1.6 

(.03) 

1.7 

86  2. 
(.97) 

Increase 

40 

(.70) 

2.1 

1.2  Reduce  "non-parking"  space  in  lots 

1.2.1  do  away  with  "no  parking"  zone^ 

1.2.2  do  away  with  extra  curb  space  in  lots 


1.5  Segregate  Ccu:s  according  to  size 

*ack  the  cars  in  more  tightly  by  painting 
closer  together 

1.7  Remove  30  minute  parking  meters 
Increase  available  space  for  parking 

2.1  Convert  more  University  land  into  parking 

2.1.1  At  the  dorm 

2.1.2  On  tie  South  Oval 

2.1.3  At  tie  Duck  Pond 

2.1.4  On  the  grass 

2.1.5  Convert  University  Streets  to 

2.1.6  At  Lloyd  Nd:>le  (basketteU)  St 

2.1.7  ^  tearing  dcwn  old  buildings 

2.1.8  By  poking  on  top  of  buildings 

2.1.9  ^  parking  inside  the  football  stadiun 

2.2  Buy  or  use  non-University  land  to  create  more  parking 

2.2.1  Buy  adjacait  houses^  convert  to  lots 

2.2.2  UEfe  adiao&it  shopping  center  at  niwt 

2.2.3  Reserve  streets  around  University  for  University 
carking 

2.2.4  Use  areas  around  fraternities  as  University  Parking 

2.2.5  Use  church  parking  lots 


26 

(.67) 


2.3  Build  parking  structures 

2.3.1  underground  stuctures 

2.3.2  high-rise  parking  structures 

2.4  All  future  buildings  should  be  hi^rise  so  that  vaoce  land 

around  them  will  be  available  for  parking 


40 


6 

(.13) 


25 

(.53) 


59 

(.83} 


11 

(.23) 


[.3 


(.07) 


5 

(.17) 


(.30) 


3.  Reduce  demand  for  parking 


3.1  Restrict  number  of  cars  allowed  to  park 

3.1.1  by  lottery:  those  who  lose  ride  motorcyles 

3.1.2  allow  oily  one  car  for  every  three  stiiclaitE 

3.2  Outlaw  cars 


3.2.1  make  everybody  ride  bikes 

3.2.2  make  everybo(^  ride  bikes  or  motorcycles 

3.2.3  for  faculty  and  staff  except  at  Lloyd  Noble 

3.2.4  except  for  cars  used  in  carpooling 

3.2.5  make  conmuter  students  park  at  Lloyd  Noble  and 

use  the  trolly  to  Haih  Campus 

3.2.6  for  employeeB:  require  them  to  take  cabs 

3.2.7  except  for  oonmuters 

3.2.8  of  stud^ts  who  live  in  Norman 

3.2.9  of  Fraternity  manbers  from  Campus  and  have  them 

park  at  their  Frats 

3.3  Introduce  or  encourage  alternative  forms  of  transportaticm 

3.3.1  widespread  rapid-transit 

3.3.2  busing  for  surrounding  area 

3.3.3  encourage  mopedSf  bicycles,  and  motorcycles 

3.3.4  people  who  live  within  one  mile  are  asked  to 

walk  or  ric3e  bikes 

3.3.5  reduce  bicyle  thefts  to  increase  bike  riding 

3.3.6  build  bicyc4e  paths  to  increase  bike  riding 
.7  sell  Icw-pricea  bicycd.es 
.8  ctarpooling  for  students 

.9  provide  tr - — 

.10  improve  " 

.11  get  a  ti _  _ ^  _ 

.12  store  dorm  residents 


ii 


3. 


a  real  trolley  c::au:  cxi  a  trac:k 

■a  t^nrm  roaii^ant-a’  rarg  Off— CSmpUS  Wlth  bUS 


to  their  cars 

3.3.13  bus  service  to  shopping,  eitertairrooit,  etc.  would 
reduc:e  need  for  cars 

3.3.14  promcTte  the  use  of  roller-skates 

3.3.15  build  sidewalks  and  improve  security  on  Czmpus 

3.4  R£d.ses  or  incentives  for  employees  or  students  who 

3.4.1  cion't  drive  to  school 

3.4.2  c^pool 

3.4.3  r\ii  to  work 
3.4.3  ride  motorcycles 

3.5  Make  it  fashionable  not  to  drive  to  work 

3.5.1  Form  "Volksmarch"  clubs  to  encourage  walking 

3.5.2  Form  cycling  clubs 

3.6  Educ:ate  people  about  conmuting  alternatives 

3.6.1  advertise  advantages  of  bikes  and  motorcycd.es 

3.6.2  educate  students  in  cx>n8ervin9  gasoline 

3.6.3  to  ln¥>rove  enviormental  quality 

3.7  Encourage  students  not  to  bring  their  cars  to  CempuB  from  home 

need  them 


3.7.1  unless^ they 

3.7.2  when  they  ar  _ 

3.7.3  if  they  live  on  campus 


real! 
e  Prei 
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3.8  Build  moving  sidewalks 

3.9  Reduce  peak  demand  for  parking 

3.9.1  Schedule  classes  differently  to  reduce  demand 

3.9.2  Have  enplpyees  work  alternate  days 

3.9.3  Have  en^yees  drive  only  on  alternate  days 

3.9.4  Let  CGRinuters  plan  their  schedules  first  so  that 
th^  can  carpool 

3.9.5  Hake  cciiinuters  stagger  class  schedules 

3.10  Increase  the  fees  for  parking  permits 

3.11  Build  more  cn-canpis  housing  for  students 

3.12  Offer  more  correspondence  courses  so  students  will  stay  at 

home 

3.13  Impound  cars  of  all  violators  of  parking  regulations 

3.14  Raise  parking  fees  to  reduce  demand 


3.15  Build  Campus  in  direction  of  open,  unused  areas  so  more 
parking  will  be  available  in  the  future 

4.  Change  parking  priorities 

4.1  Do  away  with  restricted  parking  (decaled) 

4.2  Let  people  peurk  anywhere  they  want  to 

4.3  Allot  a  i^cific  place  for  each  student 

4.3.1  based  on  seniority 

4.4  Reduce  service  vdiicle  parking 

4.5  Reduce  faculty  parking  to  increase  student  parking 

4.6  Do  not  allow  GAs  to  park  in  faculty  parking 

4.7  Hake  times  when  Eaculty^Staff  parking  is  open  more  flexible  B2 

5.  Provide  faculty  and  steiff  with  housing  within  walking  distance 

5.1  on  top  of  the  place  where  they  work 


61 

(.60) 

(.03) 


6.  Frivolous  and  other  responses 

6.1  Hove  the  University  to  the  students  by  establishing  branch 
canqpuaes 
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(.03) 

6 

(.20) 

4 

(.13) 

(.03) 

2 

(.07) 

(.03) 

7 

(.13) 


3 

(.07) 


7 

(.13) 

(.03) 

(.03) 

(.03) 

2 

(.07) 

(.03) 

3 

(.10) 


(.03) 

(.03) 

4 

(.03) 


6.2  Encourage  more  enrollment  of  foreign  studeits  (because  not 

as  many  own  cars) 

6.3  Reduce  student  population 

6.3.1  make  OU  an  "all  girls”  school 

6.4  Have  a  helicc^Jter  pick  everybo<^  iqp  for  work 

6.5  Increase  the  enrollment  cost 


6.6  I  would  have  someone  towed  away  to  create  a  qace  for  me 

6.7  Counterfeit  Faculty  parking  stickers 

6.8  Destroy  "useful”  areas 

6.8.1  tear  down  Owen  Field  (Football  Stadiun) 

6.8.2  destroy  OU 

6.8.3  destroy  Frat  Houses 

6.9  Do  aw^  with  current  University  progr2nis  to  create  parking 

6.9.1  Football 

6.9.2  Oklwoma  College  of  Continuing  Education 

6.10  Build  subway 

6.11  Destroy  cars 

6.11.01  threaten  to  destroy  cars 

6.12  Fire  teachers 


6.13  Take  aw^y  parking  from  other  branches  of  the  University 

6.13.1  Post  Office  Training  Program 

6.14  Rambling,  incoherent  thou^ts 

6.15  Let  only  people  who  have  paid  for  cars  drive 

6.16  Don't  let  people  who  attend  drive 

6.16.1  allow  only  walkers  to  attend  University 

6.16.2  make  everybo^  live  on  campus 

6.17  Have  your  parents  drive  you  to  school 

6.18  Make  everybo^  walk  who  has  less  than  a  2.3 


6.19  Impossible  acts 


ssue  everybo^  9  pair  of  wings 
ssue  strength  pills 

I  them  to  schopl^  (in  a  matter  transporter) 
reducing  or  shrinking  cars 


use 


(.03) 
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(.03) 

(.&) 


(.03) 

(.03) 

(.03) 

1 

1.03) 

(.^3) 

(.J3) 

(.03) 

(.03) 

(.03) 


6.20  Discourage  use  o£  cars  by 

6.20.1  saying  that  the  Surgeon  General  has  found  cars 

cancerous 

6.21  Send  minorities  to  Coitral  State  University 

6.22  Iranian  (some  of  these  data  were  collected  when  the  American 
hostages  were  returned  from  Iran) 

6.22.1  Nuke  Iran 

6.22.2  Take  Iranians  hostage  until  they  give  OU  money  for 

parking  lots 

6.23  Let  <»U.y  ladies  drive 

6.24  Have  ever^xx^  drive  golf  carts 

6.25  Ban  all  foreign  cars 

6.26  Have  computerized  lot  that  destroys  all  cars  exceeding  limits 

6.27  Use  area  we  have  now  for  parking  instead  of  building  the  new 

facilities  that  are  planned 

6.28  Have  harder  working  enployees  so  you  don't  need  so  many 

6.29  Only  hire  people  in  wheel  chairs 

6.30  Only  hire  people  who  like  to  rm 

7.0  Responses  that  combine  two  or  more  options  from  above 


*  The  number  of  acts  in  each  limb  or  branch. 


The  proportion  of  subjects  who  gave  limb  or  branch. 


(  tree  with  act  numbers  on  June  81  qrt.  report  disk.  This  tree  on  July  81  TR  disk.) 
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APPENDIX  C 

Act  Structure  for  Living  Problan 
1.  Live  somewhere  without  paying  rent 


1.1  With  person  he  knows 

1.1.1  With  me 

1.1.2  Wj.th  frieids 

1.1.3  With  parents 

1.1.4  Have  his  p^ents  move  here 

1.1.5  With  relatives  in  Norman 

1.2  With  person (s)  he  doesn't  knew 

1.2.1  In  fraternity  house 

1.2.2  With  persgn  of  cxnosite 

1.2.3  Somec^  with  extra  room  in  apartment 

1.2.4  Relatives'  friends 

1.2.5  With  experimaiter 

1.2.6  With  religious  cult 

1.2.7  Wiw  sponsor 

1.3  Public  institution 

1.3.1  Salvation  arm 

1.3.2  Police  station 

1.4  Miscellaneous  places 

1.4.1  Tent 

1.4.2  Streets 

1.4.3  Ogtside 

1.4.4  Hisc.  school  buildings 

1.4.5  Car  ^  ^ 

1.4.6  Trash  dunpeter 

1.4.7  Qipty  house 

1.4.8  Hotel  laixidry  room 

1.4.9  Extra  dorm  room 

1.5  Pay  for  room  only,  get  board  free 


2.  Exchange  goods  or  services  for  money  or  place  to  live 


2.1  He  could  exchange  services  for  place  to  live 


2.1.1  With  me 

2.1.2  Wiw  someone. in  town 


2.1.3  Could  be  Res 


Assistant 


2.2  He  could  exchange  goods  or  services  for  money 

2.2.1  Sell  plasma 

2.2.2  Write  bMk  or  articles 

2.2.3  Sell  paintings  or  photographs 


[ntings  or  photographs 

^ssions 
jpons  to  sell 


2.2.6  Qip  coupons  to  sel 

2.2.7  Have  carnival 

2.2.8  Sing  or  juggle  in  8l 

2.2.9  Culect  beef  cans 

2.2.10  Form  punk  rock  band 


ju^e  in  street 
Beef  cans 


2.2.11  Give  benefit  concert  for  self 

2.2.12  Enroll  in  workrstudy 
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104  3.  Obtain  money  throu^  other  sources 

(.90) 


33 

(.70) 


19 

(.33) 


3.1  Try  to  get  loan 


m 

m 


From  friends 
From  bank 
^om  loan  shark 
Tnrou^  me 
From  me. 

From  relatives 
From  private  sponsors 
From  Averment 


3.2  Apply  for  financial  assistance 


3.2.1  For.hoysir 

ilili  ifm  goy^rmmit 

3.2.4  From  school 

3.2.5  Enroll  as  foreign  exchange 


For  housing 

^ply  for  scholarship 

From  goverrmmit 


33 

(.70) 


19 

(.27) 


3.3  Ask  for  money 

3.3.1  From  other  relatives 

3.3.2  From  my  parents 

3.3.3  From  nls  parents 

^  a§  ?n*'pa^r  to  ask  for 

3.4  Donations 

.4.1  Monetary  donations  from  me 
.4.2  Ot  living  quarters 
.4.3  Fund-raising  projects 
.4.4  Take  ip  collection 


3 

1 

3 


16 

(.27) 

9 

(.17) 


(.0?) 

(.03) 

(.0?) 


4.  Ask  someone  for  suggestions  or  help 


4.1  Someone  you  know 

4.1.1  Professor 

4.1.2  Friend 

1.1.3  Relative 
.1.4  ^  parents 
4.1.5  wyrriaid's  mother 

4.2  School  officials 


4.3  Roonnate  agency 

4.4  Gonmuiity  organization 


5  5.  Try  to  get  regulations  changed 

( .17) 

5  5.1  on  working 

(.17) 


student 


contributions 
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30 

(.60) 

6. 

Change  current  plans 

2 

(.07) 

6.1  Wait  to  go  to  school  till  gets  more  money 

5 

(.17) 

6.2  Take  fewer  hours 

3 

(.10) 

6.3  Go  to  less  expensive  school 

(.03) 

6.4  Conmute  from  Canada 

(.03) 

6.5  Eat  lessr  cheaper  food 

(.03) 

6.6  work  in  Mexico  for  sanester 

(.03) 

6.7  Break  law  so  will  be  deported 

14 

(.43) 

6.8  Go  home 

5 

(.13) 

7. 

Prepare  in  advance 

(.oi) 

7.1  i^ly  for  citizenship  149 

2 

(.07) 

7.2  Invest  money  134rl35 

(.0^) 

7.3  Establish  resideicy  before  coming  146 

(.03) 

7.4  Pay  bills  so  have  good  credit  rating  148 

29 

(.40) 

8. 

Long  shots 

5 

(.13) 

8.1  Look  for  money 

8.1.1  Gold  49  __ 

II  I§  W?t^!*’raetal^ietector  127 


9 

(.10) 


8.2  Steal  money 

8.2.1  Rob  bank  36 


8.3  Get  money  through  questlonabler  illegal  methode  37 


Aaaassinate  relatives .98. 
Have  old  perscm  put  nin  in  w: 
Sell  sister  to  wnite .slaver 
Get  hit  by  car  >  si»  dri' 

JUm  off  Dale  aall  b 
Get  a  job  anyway  123 
Form  own  non-profit  oorp  155 


^ver  i2b 
attract  attention  129 
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(.ij) 


(.03) 

(.oi) 

4 

(.10) 

(.03) 

2 

(.07) 

(.03) 


8.4  Get  money  throu^  luck 

gowning,  get  reward  131 

8.5  Pre^  for  solution  to  problem  144 

8.6  Incur  revolution  147 

9.  Acts  which  will  not  solve  problem 

9.1  Conmit  suicide  150 


9.2  I  could  kill  him  152,154 

9.3  I  would  plead  guilty  to  his  illegal  activities  153 
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APPQn)IX  D 

Tree  derived  frco  OGn{d.ete  linlcage  cluster  analysis  for  Parking  Problon 

31  1.0  Do  more  with  university-owned  i^oe. 

(.80)  ^ 

20  1.1  Structures  designed  to  aid  parking  prohlen 

1.1.1  Build  hi^  rise, 

1.1.2  Build  new  buildings  as  high  rises. 

11  1.2  Reorganize  current  parking  areas 

(.30) 


32 

(.53) 

5 

(.13) 


12 

(.37) 


4 

(.10) 


11 

(.20) 


36 

(.60) 

13 

(.43) 


.  certain  areas. 

Il2l2  H^e  eh6u^~s^ces  Jot  wlryc^^Si^ng^n^^OTM. 

1.2.4  Raise  parking  fees  to  improve,  esqpand  lots. 
Lmrove  conation  of  facilities. 

1.2.5  R£»iove  30  minute  meters. 

2.  Expand  parking  to  surrounding  eureas 

2.1  Create  more  parking  space  in  surrounding  areas 

2.1.1  Put  parking  si 

2.1.2  Builolots  : 

2.1.3  Look  for  avt _ 

new  facilities. 

2.2  Build  more  lots 


ling 


2.2.1 


id  Lloyd  Ndile  parking  facility, 
id  present  areas. 


^ _ bre  lots, 

_ Id  another  lot  like  Lloyd  Noble. 

Build  more  places  like  Ldoyd  Noble. 
2.2.4  Build  a  new  lot  by  the  3oac  dorms. 

2.3  Change  parking  status  in  certain  areas 


2.3.1  Make  some  faculty  puking  student  parking. 

2.3.2  Regulate  staff  puking  hours  more  closely. 

2.3.3  Use  QU  v^icle  pirking  for  stunts. 


2.4  Use  other  areas  for  parking 

2.4.1  Dcxi't  have  so  many  no  parking  zones. 


2. 4 *4  Use  areas  around  frats  for  overflow  parking. 

2.4.5  Use  oiurch  lots. 

2.4.6  Hake  frat  members  park  at  their  own  houses. 
3.  Use  alternate  forms  of  transportation 


3.1  Carpooling 

3.1.1  Encourage  carpooling  oonnuters. 

3.1.2  OU  could  organize  a  carpool, 

3.1.3  Provide  greater  inc&itives  fu  ceurpooling. 
Reoice  price  stidcers  for  carpools. 

3.1.4  Ask  oonnuters  to  carpool. 

Use  carpools. 
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17 

(.37) 


34 

(.77) 

6 

(.13) 


28 

(.73) 


3 

(.10) 


97) 


3.2  Encourage  people  not  to  drive. 


1.2.1  Enoouri 

(.2.2  ' 


age  p^le  not  to  bring  cars, 
stgd^^s  ,about  p^Rif^  problem. . 


:e  more  Knowle^'al^t  bgnliufi^  ™^lalM>e., 

courses  available. 


3.2.3  Have  more  selection  of  evening  ( 
3.3  Promote  other  forms  of  traneportation. 


3.3.1  Advertise  the  advantages  of  riding  a  bike 
or  motorcycle. 

More  stud^ts  could  ride  bikes. 

3.3.2  Form  (^cling  clubs  for  conmuters. 

3.3.3  Encourage  students  to  stop  pollution 
riding  Dikes. 

Ask  all  stud^ts  to  ride  bikes. 

3.3.4  Build  bike  paths. 

3.3.5  Provide  more  security  for  bikes. 


4.0  Park  elsewhere^  get  other  transportation  in. 


4.1  Force  members  of  university  connunity  to  park  elsewhere. 

4.1.1  Mandatory  parking  for  facul^,  staff  at  Lloyd  Noble. 

4.1.2  Have  employees  park  at  Lloya  Noble,  use  trolly. 

1.1.3  Have  conmuters  park  at  Lloyd  Noble,  use  trolley. 

.1.4  Make  all  conmuter  parking  at  Lloyd  Ndsle. 

4.2  Biooura^  parking  in  other  areas  I:y  making  rapid  transit 


4.2.1  Set  up  rapid  transit  system  during  school  hours. 
Set  tp  bus  system  for  surrounding  area. 

Set  up  bus  ^stem  for  surrounding  communities. 
Set  up  mass  transit  system  for  Norman. 

4.2.2  Enact  bus  system  from  canpus  to  shopping  malls. 

4.2.3  Improve  trolley  and  CART  systems. 

Provitte  more  trolleys  and  busses. 

Provide  transportation  from  South  campus. 

5.  Impound  cars  without  stickers. 


6.  All  acts  having  zero  or  lower  utility  which  were  not 
a  part  of  the  cluster  analysis 


(On  disk  146) 
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APPENDIX  E 
PARKING  PROBLEM 

TREE  DERIVED  FROM  CLUSTER  ANALYSIS 
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any  act  occurring  to  them.  In  the  second  experiment,  the  acts  suggested  were 
evaluated  by  different  subjects  for  feasibility.  In  a  third  experiment,  ad¬ 
ditional  subjects  estimated  the  utility  of  the  acts  judged  feasible.  The 
act  generation  performance  of  subjects  was  evaluated  using  two  techniques. 
First,  a  decision  tree  was  generated  by  the  experimenters  by  combining  the 
acts  suggested  by  all  subjects.  The  decision  tree  generated  by  each  subject 
was  compared  with  the  experimenter- generated  tree.  It  was  found  that  sub¬ 
jects  failed  to  generate  important  limbs  and  branches  of  the  group  decision 
tree.  Second,  the  quality  of  the  trees  generated  by  individual  subjects 
was  evaluated  by  an  opportunity  loss  calculation.  This  calculation  provides 
an  estimate  of  the  potential  cost  of  failing  to  generate  limbs  and  branches 
of  the  decision  tree.  The  opportunity  loss  analysis  suggested  that  the 
failure  to  generate  a  complete  tree  could  be  costly. 
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